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USMEPCOM  Overview/History 
Purpose 

Define:  What  is  Important? 

-  Team  Charter 

-  Cause/Effect  Summary 

-  Customer  Survey  Results 

Measure:  How  are  we  doing? 

Analyze:  What  is  wrong? 

Improve:  What  needs  to  be  done? 

Control:  How  do  we  guarantee  performance 
Way  Ahead 

Questions  _ 
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USMEPCOM  Overview 


Mission 

Ensure  the  quality  of  military  accessions 
during  peacetime  and  mobilization 
in  accordance  with  established  standards. 

Organization 


Vision 

USMEPCOM  is  recognized  as  a  customer-centered, 
future-focused  learning  organization  driven  by  best 
business  practices  and  cutting-edge  technology, 
providing  real-time  entrance  processing  and  qualification. 


Syracu! 


^HQ,  USMEPCOli 
HQ,  Eastern  Set 


fQ,  Western  SecfOmah; 


p-Okiahoma  C 


$an  AntonHbuslSfi' 


i  Tam  pa 


Honolul 


Lackland  AFB 


Puerto  Rico 


-HQ’s  co-located  at  Great  Lakes  Naval  Training 
Base 

-2  Sectors 

-65  Military  Entrance  Processing  Stations  (MEPS)y 
-Supports  all  uniform  services 

Resources 

Budget:  $195  million  annually 


Western  Sector 


Authorized 


Total  Military:  1405 
Total  Civilian:  1386 


Assigned 

1480 

1275 


Facilities:  67  Across  the  Nation 
costs:  30%  of  budget 


^rtlar 
istor 

Detroi  ^ffafo 


-Ft.  Dix 

Pittsburg 

Columbus  Batimorelrf^n-^—  _ 
dianapolls 

Louisv|lle^ichm^f 

,  Knoxville  charloSl'®'®'’ 
ash^lle  ^ 

Montgorrvry 
lackson 


f  :^ams  Island 
Jacksonville 
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Introduction 


•  How  accurate  is  your  data? 

•  How  do  you  Know? 

•  This  brief  provides  a  Six  Sigma  methodology  used  to 
answer  this  question. 


5 


Purpose 

•  Provide  Information  on  USMEPCOM’s  Data  Quality  Metric. 

•  Initial  Request  from  Office  of  the  Under  Secretary  of  Defense  for 
Personnel  &  Readiness  Information  Management  (OUSD  P&R  IM). 

-  Focus  area:  Establishing  and  reporting  measurements  for  capability-level 
metrics. 

-  Collect  metrics  to  report  to  Congressional  Report/Enterprise  Transition 
Plan.  One  of  the  metrics  being  collected  is: 

•  Accuracy  of  Accession  Information  -  This  is  defined  as  the  percentage  of 
accessions  that  are  accurately  tracked. 

•  Can  USMEPCOM  provide  information  on  how  you  are  currently  collecting  the 
above  information? 

•  For  example,  what  are  the  data  sources  (system/initiative,  reports,  other 
sources)?  How  often  are  they  reported?  How  hard  are  they  to  collect?  What 
formulas  are  utilized  in  the  capturing  and  recording  of  metrics. 

•  Define  and  implement  USMEPCOM’s  data  quality  metrics  to  ensure 

data  quality  for  the  accessions  community. 

-  How  accurate  is  our  data?  How  do  we  know? 

-  What  metrics  do  we  use  to  track  data  quality? 

-  Who  monitors/tracks? 

-  Who  is  responsible? 
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Facts 


•  USMEPCOM  requested  to  provide  data  quality  metrics  for 
quarterly  reports  to  Congress  by  P&R  IM. 

•  USMEPCOM  had  no  existing  capability  to  measure  and 
report  the  Quality  of  accession  data. 

•  USMEPCOM  Data  Quality  Team  used  a  Six  Sigma 
approach  to  established  data  quality  metrics. 
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Assumptions 


•  It  is  in  USMEPCOM’s  strategic  and  operational  interest  to 
be  able  to  measure  and  report  on  the  quality  of  key 
mission  and  mission-support  data. 

•  OSD  will  continue  requesting  that  USMEPCOM  provide 
data  quality  information. 

•  USMEPCOM  is  able,  within  current  funding  and  staffing 
levels,  to  design  and  implement  a  practical,  effective,  and 
efficient  capability  to  measure  and  report  key  elements  of 
data  quality. 

•  USMEPCOM’s  long  term  value  to  the  accession  triad  is 
data. 
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Team  Charter 


Project :  improve  USMEPCOM  Accession 

Data  Quaiitv 

Business  Case 

•  Improve  USMEPCOM  data  quality,  and  establish  a 
data  quality  standard. 

•  Data  Quality  is  a  function  of  accuracy,  validity, 
timeliness,  and  transmission  reliability 

•  Communicate  to  stakeholders  USMEPCOM’s  data 
quality. 

Goal  Statement 

Improve  USMEPCOM  monthly  data  quality  to  reach 
standard  of  99%. 

Project  Plan 


Opportunity  Statement 

•  Improve  credibility  with  stakeholders 

•  Improve  MEPS  business  process 

•  Improve  USMEPCOM  relevance  as  data  providers 

•  Increase  effective  communication 

•  Track  business  rules/data  collection  changes  over 
time 

•  Establish  data  warehouse 

•  Improve  data  business  rules 

Project  Scope 

•  Focus  on  Accession  data 

•  Identify  key  data  fields  (Stakeholder  Focus) 

•  Focus  on  fields  that  create  DEERS  record 

•  Best  business  practices  (HMO’s...) 

•  Policy  implementation 

•  Data  accountability 


Task/Phase 

•  Phase  1:  Initial  Response  to  P&R  IM  NLT  02JUN 

•  Phase  2:  Develop  USMEPCOM  data  quality  metric 

•  Finalize  Charter;  NLT  02MAY06 

•  Gather  Data:  NLT  25SEP06 

•  Analyze  Data:  NLT  01OCT06 

•  Select  Solution:  NLT  01NOV06 

•  Develop  Plan  to  implement:  NLT  30NOV 
Closure  and  Recognition:  NLT  31  DEC. 


Team  Members 

Champion:  Mr.  O'Brien 

Black  Belts:  MAJ  Mark  Gorak  /  Rick  Cox 

Process  Owner :  J5 

Member:  J3:  Don  Hill/Theresa  Morris/Tom  Buehring 
Member:  J6:  Bill  Armstrong/Bernard  Underwood 
Member:  J7:  SMSGT  Dean  Dzurovcin 
Member:  J8:  Rich  Harris/Mike  Arsenault 
Member: 
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Critical  Customer  RequirementsSS®®® 
Survey  of  Key  Stakeholders 

Conducted  17-Question  Survey  of  DADSIWG  members. 

•  How  would  you  rate  the  ACCURACY  of  USMEPCOM 
data? 

—  50%  of  respondents  felt  accuracy  was  90%  or  less,  or  didn’t  know 

•  How  important  is  it  to  the  accession  community  for 
USMEPCOM  data  to  be  accurate? 

—  100%  said  “Extremely  Important” 

•  Comments 

—  “Senior  leaders  make  policy  decisions  based  on  the  data  we  get 
from  USMEPCOM.  If  inaccurate,  decisions  made  are  faulty  and  can 
have  adverse  political  and  financial  impacts.  ” 


—  “SECDEF  and  other  senior  ieaders  expect  and  want  timely  and 
up  to  date  answers  to  their  questions.  Today  we  will  have  to  wait  a 
week  or  so  for  the  data  to  be  loaded  is  totally  unacceptable.” 
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Definitions 


>D))M 


m: 


Data  Quality:  A  function  of  accuracy,  validity,  timeliness,  and 
transmission  reliability. 

•  Data  Accuracy: 

-  Conformity  to  fact.  Exactness.  The  ability  of  a  measurement  to  match  the 
actual  value  of  the  quantity  being  measured. 

-  Accuracy  refers  to  whether  the  data  correctly  records  the  business  object  or 
event  it  represents.  It  has  two  requirements:  it  must  be  the  right  value  and  it 
must  represent  the  value  in  a  consistent  form  with  all  other  representations  of 
the  same  value. 


•  Data  Validity: 

-  Used  to  find  invalid  data,  not  all  inaccurate  data. 

-  Data  entered  into  data  field  corresponds  to  a  valid  domain  range  value  for  that 
field.  (Address  field  has  valid  address) 

-  Data  Representation:  A  text  field  containing  the  name  of  a  state  may  have 
entries  for  TX,  TEXAS  and  texas.  To  someone  looking  at  a  report,  this  might 
not  be  a  problem,  however,  to  data  queries  this  poses  several  problems. 


•  Timeliness: 

-  Available  to  data  users  when  and  where  needed. 

-  Data  is  reported  within  prescribed  timelines. 

•  Transmission  Reliability: 

-  Data  element  is  complete;  entered  into  system  and  is  accounted  for  from  data 


Data  Quality  Factors 


Context 


Continuous 

Monitoring 


Storage 


Stewardship 


Goa  S 


Decisions 
Actions 


Data  Row 


Work  Flow 


Resources 
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SIPOC 


USMEPCOM  Accession  Data  Flow 

Start  Boundary:  Record  Build  into  MIRS 

End  Boundary:  USEMPCOM  sends  accession  data  to  DMDC 

Suppliers 

Inputs 

Outputs 

Customers 

Recruiters 

MEPS  -  MIRS 

Applicant  Accession  Data 

Services 

Applicants 

Service  -  MIRS  (AF,  Navy,  CG) 

Applicant  Information  (15K 

DMDC 

MEPS 

MCRISS  /  ARISS 

CO 

e/\ 

QuICR 

Directorates 

SSA 

SSA  (response) 

UJ 

(S 

OSD-AP 

DMDC 

DMDC  (Prior  Service) 

o 

SSA 

Lab's 

Drug  Lab 

Q. 

FBI  (BK  Check) 

HIV 

STEPS: 

See  next  slide:  US  MIRS  Data  Flow 
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Cause  -  Effect  Diagrar^®®®P 


People 


Machines 


Data  input, 
RMS) 


Warehouse 


I  Environment 

tA/gfKtotjyr^oU  j 


IG  Checks 


rheck  QRP 

A/orka  rounds 


Turnover 


.  CKa/%jC 

'\ArAwA  r<  /SoF.(  u/<  I  ^  iAi  SS  ‘OA  Pr  e At 
Databases  fxTs 


META  Data 


T rack  Defects 

A/o 

^standard  Repot^^ 


Measurements 


Business  Rules 
Training 

- III  I  iTi' 
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Methodology  Results:  Cause/Effect  Summary 


1. 


4. 


5. 


8. 

9. 

10. 


High  Direct  Impact  on  Data  Quality 
System  Interfaces:  (#  of  transmission  failures  & 
validity  of  edits  between  systems  {Need 
feedback  from  interface  partners)) 

Tracking  Defects:  #  of  Mismatches  (POB,  DOB, 
SSN,  USCIS  code,  etc.) 

External  Requirements:  #  of  Change  Requests 
per  month. 

Data  Definitions:  Incorrect  definitions  or 
changes  to  Data  Def 
Data  Input  Methods:  #  of  fingerprint  error 
(mismatch/error  with  0PM) 

Manual  Data  Entry:  Keystroke  errors,  deception, 
etc. 

Business  Rules  &  Processes:  (#  of  validated 
Requirements,  incorrect  data) 

Workarounds:  #  K-Deletes,  etc. 

Data  Quality  from  Services:  (Missing  data, 
incorrect  interface  edits?) 

Staffing  Level:  (#  of  personnel,  #  of  MIRS 
terminals,  #  trained,  quality  &  #  training 
resources) 

stem  Reliability:  ^^f  transmission  failures  & 
‘a  timeless) r  "  *  ^ 


Indirect  Impact  on  Data  Quality 

1 .  Measurement  System 

2.  Database  Architectures 

3.  Data  Warehouse  (Discovery) 

4.  Standard/adhoc  Reports 

5.  Obsolete  Hardware  &  Software 

6.  User  Interfaces  (Edits  & 
correction  protocols) 

7.  Formal  Processes  & 
Procedures/Business  Rules 

8.  System  Oversight  (roles  and 
responsibilities) 

9.  Forms  Design/data  capture 

10.  Regulations/Formal  guidance 

1 1 .  Personnel  T urnover  -  Formal 
training 

1 2.  KSAs  -  Job  description 
accuracy 

1 3.  Mission  Pressure  -  Support  for 
your  customers 

14.  Facilities 

15.  System  User  Interface 
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Key  Data  Quality  Metrics 


Metrics  Considered  and  Metrics  Considered 

Recommended _ but  not  Recommended 

-  SSN 


-  Name 

-  Date  of  Birth 

-  Address 


-  HIV 

-  DAT 


-  K-Deletes 

-  Fingerprint 

-  System  T ransmission  Errors 
Data  Logs 

*  The  team  agreed  that  we  should  measure 
data: 

1)  on  the  DD  Form  1172-2  (DEERS 
enrollment), 

2)  reported  outside  USMEPCOM  upon 
accession  into  AC  or  RC  status, 

3)  widely  depended  upon  by 
Community  of  interest,  and 

4)  within  our  control  to  correct. 


Aptitude  Test  Score 
Place  of  Birth 
SSN  Duplicates 
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SSN  Process  Flow 
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SSN  Process  Flow  Continued 


Personal  Record  Created  in  M IRS  AOOOV 
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SSN  Process  Flow  Result  Code 

SSA  VERIFICATION  CHECK 


YES  »  Check  to  see  if 
Citizenship  matches  and  / 
or  Death  data  not  present 
in  SSA  database 


Citizen  mismatch  and  Death  data  present 
»  S8  GENDER  &  DOB  MISMATCH 


Citizen  mismatch  »  S6  GENDER  MISMATCH  WITH  SSA 


Death  data  present »  S7  DOB  MISMATCH  WITH  SSA 
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XML/sFTP 


“To  Be”  Data  Transmission  Flow 


MCRISS 


Recruiting  Services 


ARISSAk  Others^ 

Exchangfi'  ^  ^  ^ 

RlCRISS^ICRxrliarwir  ^ 

Enterprise  Service  Bus 


USMIRS  Data  Flow 

TO  BE  DEC  07 


X 


"D 


TOSIP  (SOA) 


PUSH  OR  PULL 


Applicant  Data 


External  Agency 
^  Results 


Metric  Summary 


Data 

Element 

Error 

Condition 

Error 

Test 

Quality  Metric 

Error 

Standard 

Weight 

(1-10) 

POC 

SSN/Name/ 

DOB 

SSN/Name 
mismatch 
with  SSA  file 

SSN 

mismatch 

report 

Percent  defective  SSNs  in  all  AOOOV 
files  (by  each  S  codel ,  3,  &  5)  (# 
Defects  /  Total  #SSN)  3  measures 

1% 

10 

J5 

Address 

Incorrect 

Address 

Compare  to 
MapMarker 

Percent  of  Address,  City,  State,  Zip 
accurate  compared  to  MapMarker 
(#inaccurate  /  total)  4  measures 

1% 

8 

J5 

K-Deletes 

K-Delete 

Event 

Sample 

CRDB 

monthly 

Percent  of  K-deletes  (#K-Deletes  / 
#Unique  RJD  trans) 

.001% 

5 

J3 

Fingerprints 

Unusable 
fingerprint — 
must  be 
redone 

FBI 

exception 
report  from 
0PM 
monthly 

Percent  resubmissions  required 
(#Resub  /  #Total  Submissions) 

.01% 

7 

J3 

System 

Data  Logs 

Transmission 

failure 

Analyze 
system  logs 
per  month 

Percent  of  data  transmissions  to 
external  stakeholders  that  required 
more  than  one  trans.  (  #Trans  failure 
/  Total  DMDC  Transmissions) 

0% 

1 

J6 

Each  data  element  reported  by  total  number  of  Accessions. 


J5  has  overall  responsibility  for  reporting  metrics  based  on  input  from  POC’s  listed  above. 
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Data  Quality  Metrics  Results 


Accessions  MAR  APR  MAY  JUN  3QTR06 


Data 

Element 

Goal 

WGT 

POC 

%EV 

%EV 

%EV 

%EV 

%E  V 

Data 
Quality 
Contb  * 

Percent 
of  Goal 

SSN 

0.100% 

10 

J5 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

20.833 

100.00% 

Name 

0.800% 

10 

J5 

0.85% 

0.87% 

0.83% 

0.85% 

0.85% 

20.655 

93.65% 

DOB 

1 .000% 

7 

J5 

1 .08% 

0.93% 

1.01% 

0.67% 

0.88% 

14.455 

100.00% 

House  #  + 

5.000% 

2 

J5 

22.91% 

23.02% 

22.19% 

22.47% 

22.54% 

3.228 

22.18% 

Street  +  City 
+  Zip 

4.000% 

2 

J5 

22.52% 

22.55% 

21.72% 

21.99% 

22.06% 

3.247 

18.13% 

City  +  Zip 

1 .000% 

2 

J5 

14.47% 

14.65% 

13.72% 

14.14% 

14.15% 

3.577 

7.07% 

Zip 

1 .000% 

2 

J5 

10.64% 

10.77% 

10.01% 

10.18% 

10.29% 

3.738 

9.72% 

K-Deletes 

0.500% 

5 

J3 

0.33% 

0.26% 

0.97% 

0.70% 

0.64% 

10.350 

78.19% 

Fingerprints 

0.750% 

7 

J3 

0.00% 

0.75% 

0.78% 

0.72% 

0.75% 

14.474 

100.00% 

System 

Data  Logs 

0.000% 

1 

J6 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

2.083 

100.00% 

A  96%  Data  Quality  Metric  indicates  that  USMEPCOM’s  Data  Quality  is 

Good.  (90-95%  Poor,  95-98%)  Good,  98-99%o  Great,  >99%  Outstanding)  96.64  62.8% 


*  Data  Quality  Contribution  =  [(100  -%  Error  Value)  X  Weight]  /  Sum  of  Weights 
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Daily  Data  Quality  Metric  Contribution 


Summary  for  Data  Q  Contribution 

FY  =  2007 
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Daily  Data  Quality  Metric  Contribution  )d})m»a))i))c 


The  6-pack  chart  shows  that  the  pattern  of  data  appears  random,  with  a  near 
constant  mean,  and  uniform  variability  =  stable  process. 


98.4t 


Process  Capability  Sixpack  of  Data  Q  Contribution  Need  to  s  lift 

I  Chart  CapabUit^Tflstogram^ — j^0gp|  ■fQ  pght 


53  79  105  131  157  183  209  235  261 

Last  25  Observations 


Capability  Plot 


97.6t 


U)  -S' 

S  96.8-  • 


96.0- 


Within 

StDev  0.34837 
Cp  1.44 

Cpk  -0.01 


240 


245  250 

Observation 


255 


260 


Within 


Overall 


■h — r— h 


Specs 


Overall 

StDev  0.376545 
Pp  1.33 
Ppk  -0.01 
Cpm  * 
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Daily  Data  Quality  Metric  Contribution  g>g)H>E>g) 

The  3  points  outside  the  lower  control  limit  are  due  to  a  System  Data  Transmission 
error  for  that  day  (19SEP06,  26DEC06,  and  03JAN07). _ 

USMEPCOM  Data  Quality  Contribution  Metric 

Daily  Results 


2006  2007 

98. 5  i  I 
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Way  Ahead 


•  In  MAY  2007,  OSD  AP,  formed  the  Data  Reconciliation 
Working  Group  (DRWG)  to  improve  consistency  of 
accession  data  between  USMEPCOM,  DMDC  and  the 
Services. 

•  Improve  USMEPCOM  data  quality  by  implementation  of 
automatic  address  correction  software 

•  Establishment  of  a  data  quality  measurement  system  used 
for  monthly  business  reporting  and  management 

•  Document  root  causes  of  variations  in  data  quality  and 
provide  solutions  to  decease  variations. 
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DECISION 


??  QUESTIONS  ?? 
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Measure  Mar  Apr  May  June  3QTR06  July  Aug 


SSN  (1)  ERRORS  Applicants 

46 

34 

168 

51 

253 

45 

56 

SSN  (1)  SUBMISSIONS  Applicants 

49578 

41856 

46401 

48614 

136871 

44849 

48913 

SSN  (1)  ERROR  RATE  Applicants 

0.09% 

0.08% 

0.36% 

0.10% 

0.18% 

0.10% 

0.11% 

SSN  (1)  ERRORS  Accessions 

0 

0 

0 

0 

0 

0 

0 

SSN  (1)  SUBMISSIONS  Accessions 

17991 

14518 

14171 

11814 

40503 

9636 

8144 

SSN  (1)  ERROR  RATE  Accessions 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

SSN  (1)  ERRORS  DEPS 

0-Jan 

0 

4 

0 

4 

0 

0 

SSN  (1)  SUBMISSIONS  DEPS 

13499 

11620 

12977 

15257 

39854 

13298 

12706 

SSN  (1)  ERROR  RATE  DEPS 

0.00% 

0.00% 

0.03% 

0.00% 

0.01% 

0.00% 

0.00% 

Name  (5)  ERRORS  Applicants 

980 

824 

943 

998 

2765 

937 

1152 

Name  (5)  SUBMISSIONS  Applicants 

49578 

41856 

46401 

48614 

136871 

44849 

48913 

Name  (5)  ERROR  RATE  Applicants 

1.98% 

1.97% 

2.03% 

2.05% 

2.02% 

2.09% 

2.36% 

Name  (5)  ERRORS  Accessions 

153 

127 

118 

101 

346 

85 

58 

Name  (5)  SUBMISSIONS  Accessions 

17991 

14518 

14171 

11814 

40503 

9636 

8144 

Name  (5)  ERROR  RATE  Accessions 

0.85% 

0.87% 

0.83% 

0.85% 

0.85% 

0.88% 

0.71% 

Name  (5)  ERRORS  DEPS 

98 

95 

115 

129 

339 

110 

83 

Name  (5)  SUBMISSIONS  DEPS 

13499 

11620 

12977 

15257 

39854 

13298 

12706 

Name  (5)  ERROR  RATE  DEPS 

0.73% 

0.82% 

0.89% 

0.85% 

0.85% 

0.83% 

0.65% 

DOB  (3)  ERRORS  Applicants 

577 

452 

525 

397 

1374 

343 

396 

DOB  (3)  SUBMISSIONS  Applicants 

49578 

41856 

46401 

48614 

136871 

44849 

48913 

DOB  (3)  ERROR  RATE  Applicants 

1.16% 

1.08% 

1.13% 

0.82% 

1.00% 

0.76% 

0.81% 

DOB  (3)  ERRORS  Accessions 

194 

135 

143 

79 

357 

52 

55 

DOB  (3)  SUBMISSIONS  Accessions 

17991 

14518 

14171 

11814 

40503 

9636 

8144 

(3)  ERROR  RATE  Accessions 

1.08% 

1 _ 

0.93% 

1.01%  , 

■^-0^% 

0.54% 

0.68°^ 

_ 

loi^"' 

jjDOj  (3)  ERRO^  DEPS  ^ 

.  —  ^ 

JUmmmWswx 

368 

(3)  SUE^SSIUNS 

1525t-aa 

132^ 

127(^ 

DOB  (3)  ERROR  RATE  DEPS  1 .06% 

1.04% 

1.05% 

0.73% 

0.92% 

0.71% 

0.84% 

